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Abstract — A fundamental property of codes, the second-order 
weight distribution, is proposed to solve the problems such as 
computing second moments of weight distributions of linear 
code ensembles. A series of results, parallel to those for weight 
distributions, is established for second-order weight distributions. 
In particular, an analogue of MacWilliams identities is proved. 
The second-order weight distributions of regular LDPC code 
ensembles are then computed. As easy consequences, the second 
moments of weight distributions of regular LDPC code ensembles 
are obtained. Furthermore, the application of second-order 
weight distributions in random coding approach is discussed. 
The second-order weight distributions of the ensembles generated 
by a so-called 2-good random generator or parity-check matrix 
are computed, where a 2-good random matrix is a kind of 
generalization of the uniformly distributed random matrix over 
a finite filed and is very useful for solving problems that involve 
pairwise or triple-wise properties of sequences. It is shown that 
the 2-good property is reflected in the second-order weight 
distribution, which thus plays a fundamental role in some well- 
known problems in coding theory and combinatorics. An example 
of linear intersecting codes is finally provided to illustrate this 
fact. 

Index Terms — Low-density parity-check (LDPC) codes, 
MacWilliams identities, random linear codes, second moments, 
weight distributions. 



I. Introduction 

THE weight distribution is an important property of a code. 
Knowledge about the weight distribution of a linear code 
is very useful for estimating decoding error probabilities lfT9l 
and covering radius |6|, etc. 

Let ¥q be the finite field of order q ~ p^, where p is prime 
and r > 1. Throughout this paper, all codes considered will be 
over ¥q. For a codeword c G F" its (Hamming) weight w(c) 
is the number of nonzero symbols in c. For a code C C F" 
the weight distribution Ai (C) is the number of codewords of 
weight i in C. The polynomial Wc{x) — X]"=o ^«(^)-^* ^^ 
called the weight enumerator of C. 

In general, it is hard to compute the weight distribution 
of a specific code. On the other hand, for some ensembles 
of codes, we can compute their average weight distributions. 
For example, we have known the average weight distribution 
of various low-density parity-check (LDPC) code ensembles 

na, M, mil, m, iuti, fT9i, m, ma, jsa, mm, and even 

the second moment of weight distributions of some specific 
LDPC code ensembles, e.g., H], E), ^, ^. While the 
average weight distribution provides by Markov's inequality 
an upper bound of the weight distribution of codes in the 
ensemble with high probability, the degree to which individual 
codes have a weight distribution close to this average depends 
on second- or higher-order statistics of the random weight 
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distribution. Typically, given a random code C (a measurable 
mapping from some abstract probability space to the power set 
of F"), one computes the second moment E[{Ai{C))'^] of the 
weight distribution, or the second moment E[Wc(x)Wc{y)] 
of the weight enumerator, since 

E[{A{C)f] = coei{E[Wc{x)Wciy)],x'y') 

where coei{p{x,y),x'^y^) denotes the coefficient of x'''y^ in 
the polynomial p(a;, y). Applying Chebyshev's inequality with 
the second moment then gives a confidence interval of the 
weight distribution of an individual code with respect to any 
given probability. It is not the only case that one needs to 
compute the second moment like E[Wc{x)Wc{y)]- When 
estimating the variance of undetected error probability of an 
error detection scheme (see e.g. ||29l p. 99] and lf36l ) which 
is expressed in terms of a random weight enumerator Wc{x), 
one also needs to compute the second moment £'[(Wc (x))'^] or 
E[Ai{C)Aj{C)]. Then one question arises: how can we com- 
pute these second moments? For this question, there has been 
some work for some specific code ensembles, e.g., the second 
moment of the weight distribution of a binary regular LDPC 
code ensemble fl], fJT], ["321, the co variance of the weight 
distribution of a linear code ensemble characterized by the so- 
called Bernoulli parity-check matrix ensemble lf36l . and the 
second moment of the weight distribution of a random linear 
code generated by a uniform random generator matrix Q. 
However, no systematic approach has ever been estabhshed to 
facilitate such kinds of computation. 

To establish a systematic approach, we need a fundamental 
property of linear codes, which not only yields the second mo- 
ment of weight distribution but also supports easy computation 
for various combinations of linear codes. Unfortunately, the 
distribution Ai{C)'^ or Ai{C)Aj{C) are not qualified for this 
position. When C is not random, it is clear that E[Ai{C)Aj (C)] 
provides no more information than does £'[j4i((J)]. On the 
other hand, for a general random C, the information contained 
in E[Ai{C)Aj{C)] is too coarse to support the computation 
of serially concatenated codes, even if an analogue of input- 
output weight distribution (see e.g. ifTSl ) is introduced. Recall 
that a linear code is the kernel or image of a linear transforma- 
tion. Then most kinds of combinations of linear codes can be 
expressed as a series of two basic operations of linear trans- 
formations, namely, the composition (serial concatenation) and 
the Cartesian product (parallel concatenation). 

Motivated by the question above, we provide in this pa- 
per a novel property of codes, called second-order weight 
distributions. From the viewpoint of group actions on sets, 
the second-order weight is a partition induced by the group 
of all monomial maps acting on the set F" x F" so it 
is a natural extension of weight, which is a partition in- 
duced by the same group acting on F" A series of results. 
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parallel to those for weight distributions, is established for 
second-order weight distributions. In particular, an analogue of 
Mac Williams identities is proved. Equipped with this new tool, 
we compute the second-order weight distributions of regular 
LDPC code ensembles. As easy consequences, we obtain the 
second moments of weight distributions of regular LDPC 
code ensembles, which include the results of ||T|, |f3T| . If32l 
as special cases. Furthermore, we discuss the application of 
second-order weight distributions in random coding approach. 
We compute the second-order weight distributions of the 
ensembles generated by a so-called 2-good random generator 
or parity-check matrix. A 2-good random matrix is a kind of 
generalization of the uniformly distributed random matrix and 
is very useful for solving problems that involve pairwise or 
triple-wise properties of sequences. We show that the 2-good 
property is reflected in the second-order weight distribution, 
which thus plays a fundamental role in some well-known 
problems in coding theory and combinatorics. An example 
of linear intersecting codes is finally provided to illustrate this 
fact. 

The rest of this paper is organized as follows. In Section Ull 
we establish the method of second-order weight distributions. 
In Section |lll] we compute the second-order weight distribu- 
tions of regular LDPC code ensembles. The application of 
second-order weight distributions in random coding approach 
is discussed in Section HV] Section IV] concludes the paper. 

In the sequel, the symbols Nq, C, S„ denote the set of 
nonnegative integers, the field of complex numbers, and the 
group of all permutations on n letters, respectively. The 
multiplicative subgroup of nonzero elements of ¥q (resp. C) is 
denoted by F* (resp. C*). A vector in F" is typically denoted 
in the row- vector form v — (wi,W2, • • • ,Vn)- The canonical 
projection tt^ : F" ^- F, is given by v H> Vi. In general, 
for an element in a set A^ , we adopt a similar notation such 
as V = {vi)i^i where Vi G A, and the canonical projection 
TT,; : A' ^ A with i G / is given by v H> Vi. Given u E A^ 
and V G B^ , if the product Hie/ "^T rn^kes sense, we write 
u'^ as the shortening. For any set A and its subset B, the 
indicator function 1b ■ A ^ {0, 1} is given by a; H> 1 for 
X (£ B and a; i— > for x ^ B. When the expression of B is 
long, we write 1_B in place of 1b(x). Nonrandom codes are 
denoted by capital letters, while random codes are denoted by 
script capital letters. Matrices are denoted by boldface capital 
letters. By a tilde we mean that a matrix such as A is random. 
Unless stated otherwise, distinct random elements are assumed 
to be independent. 

II. Second-Order Weight Distributions 

In order to find a fundamental property that underlies the 
second moment of weight distributions, let us first take a close 
look at the product Ai{C)Aj{C), which may be rewritten as 



J2 E 1{(u,v)gCxC}. 



A,{C)A,{C) = 

u:w(u)— -i v:w(v)— J 

Also recall the definition of the weight distribution 

u:w(u)— 2 



It is then reasonable to guess that the fundamental property 
that we are seeking may be a sum of l{(u, v) G C x C} over 
some set of vector pairs. More specifically, let P be a partition 
of Fg X F^\ and then the quantity 

Ap{C,C)^ E 1{(u,v)gCxC} for Per 

(u,v)GP 

gives a kind of property of C. Whenever P is a refinement of 
the partition Q = {Q{iJ) = {(u,v) : w(u) = i,w(v) = j}}, 
Ap{C,C) can readily yield A,{C)Aj{C). 

One obvious choice of V is the finest partition of F" x F" 

i.e., O = {{(u, v)}}ueF",veF"- However, the partition O 
contains so much information that the complexity of induced 
formulas grows out of control as n increases. On the other 
hand, as we have shown in Section |I] the coarsest partition 
Q itself is not qualified, because it contains no enough 
information. Then our task is now to find an appropriate 
partition between O and Q. 

A similar story has ever happened on the weight distribu- 
tion. In order to find the answer, we shall briefly review the 
reason why the weight distribution is so fundamental. 

Let G be the random matrix uniformly distributed over the 
set F™'^" of all TO X n matrices over Fg. It is well known 
that the linear code ensembles {uG : u G F™} and {u G 
F;^' : Gu^ = 0} are both good for channel coding Jl], |fT9l . 
Moreover, the application of G is not confined in channel 
coding. It also turns out to be good for Slepian-Wolf coding 
1 12 1, lossless joint source-channel coding (lossless JSCC) |38| , 
and so on. 

The success of G in information theory exclusively depends 
on its fundamental property: 



P{F{u) - v} = q- 



VuGF"\{0},vG 



(1) 



where F(u) = uG. In fact, any random linear transformations 
satisfying ([T]i has the same performance as F for channel 
coding, lossless JSCC, etc. We may call such random linear 
transformations good random linear transformations. 

One important property of good random linear transforma- 
tions is that both f o F and F o 5 are good for any bijective 
linear transformations / : F^ -> F^ and g : F™ -^ F™. In 
particular, a good random linear transformation is preserved 
under a special class of mappings called monomial maps 11211 
Sec. L7]. Let c G (F*)" and a G S„. We define the monomial 
map Cc. : F^' ^ F^ by 

^c,£7(v) = (ciW^-l(i), C2V^-i(2), ■•-,£„ U^-l(„)). (2) 

Furthermore, we define the uniform random monomial map 
S„ as a random mapping uniformly distributed over the set 
of all monomial maps. Then given a linear transformation / : 
F™ ^> F" we define the randomization operator 



T^if) = ^„ o / o S„ 



(3) 



Note that, according to our convention, S,„ and S„ are 
independent. It is clear that for any good random linear 
transformation F, 



P{7e(F)(u)-v} 



Vugf™\{o},vg: 
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This implies that TZ does not deteriorate the average perfor- 
mance of the ensemble. Moreover, the new ensemble 7?,(F) 
gets larger than F and has more symmetries, which facilitate 
the analysis of codes. Proceeding with this notion, we may 
consider such a coding system, where all linear transfor- 
mations are randomized by independent operators TZ. It is 
not a new idea. Both Turbo codes ||5j and LDPC codes are 
constructed by this randomization technique, and the analysis 
of weight distributions always enjoys such code ensembles. 

Let Mn be the set of all monomial maps. Then under 
function composition, A/„ forms a group (called a monomial 
group) that acts on F"U This notion then establishes the 
relation between M„ and weight, that is, each set of vectors 
with the same weight corresponds to exactly one orbit of 
Mn on F" In other words, the weight is nothing but an 
identification of the orbits of M„ on F" This explains why the 
average weight distribution of Turbo codes and LDPC codes 
are easier to compute than other codes not randomized by TZ. 

Now that the orbits of M„ on F" induce the weight, it is 
natural to consider the orbits of Af„ on F" x F" by the action 
(^, (u, v)) H^ (C(u), C(v))- As we shall see later (Lemmas 12.71 
and 12.91 and Theorem 12.12b . these orbits give an appropriate 
partition of F" x F" between O and Q. Therefore, we choose 
this partition as the basis for defining the fundamental property 
of codes. As a natural extension of weight, it will be called 
the second-order weight^ Now, it only remains to give a 
convenient identification of the orbits. 

To make things easier to be understood, let us begin with 
n = 1. Consider the orbits of Mi (i.e., F*) on F^. There 
are totally g + 2 orbits in F^. We denote by S the set of all 
orbits. It is clear that S consists of the zero subspace and 
q + 1 one-dimensional subspaces of F^ with (0, 0) excluded. 
For each orbit S* G tS, we also denote it by v for any chosen 
representative u G S*. The standard representative p{S) of 
an orbit S" G 5 is defined as its unique element whose first 
nonzero component is 1, or (0, 0) for the special orbit {(0, 0)}. 
For convenience, we also define some special elements and 
subsets of S: 



Sm — (0,0), 5oi : 
SaQ — {Sm}, Sii 



(0,1), 5io = (l,0) 
: S \ {S'oo, 5*01, 'S'lo}- 



Example 2.1: When g = 3, we have 

5 = {{(0,0)},{(0,1),(0,2)},{(1,0),(2,0)}, 
{(1,1),(2,2)},{(1,2),(2,1)}} 
p(5)- {(0,0), (0,1), (1,0), (1,1), (1,2)}. 

'a group G is said to act on a set S, if there is a function (or action) 
Gx S —^ S such that for all x £ S and gi, g2 & G, ex = x and {g\g2)x = 
gi{g2x), where e is the identity element of G. For a given x Si S, the set 
{gx : g S G} is called the orbit of x and is denoted by x. For different 
x,y a S with 5 n y 5!^ 0, we have x = y, so all sets x for x a S form a 
partition of S. For details, the reader is referred to [22 Sec. II. 4]. 

^The second-order weight is not the unique partition satisfying our require- 
ments. If replacing A/,j with the group of all permutations of coordinates, 
i.e., the set of all ^lo- with a £ S„, we shall get a finer partition, which 
also serves our goals and may be called the second-order complete weight or 
second-order spectrum, since it is a natural extension of the complete weight 
or spectrum (see 0), 1271 . 1381 ). However, the second-order complete weight 
is much more complex than the second-order weight. 



Having introduced the basic notations for identifying every 
orbits of Ml on F^, we are now ready to formally define the 
second-order weight and all related concepts. Our approach 
simply follows a similar way in which the weight as well as 
the weight distribution is defined. 

Definition 2.2: For any u, v G F" the second-order weight 
of (u, v) is a (g + 2)-tuple defined by 



W2(U, 



^'^siui 



ses 



With this definition, it is easy to verify that the second-order 
weight parametrizes the orbits of the monomial group 7\f„ on 
F" X F" The next lemma formally states this fact. 

Lemma 2.3: If the monomial group M„ acts on F" x F" 
by the action (^, (u, v)) 1— > (^(u),^(v)), then the orbit of 
(u, v) G F" X F" is exactly the set of all vector pairs of 
second-order weight W2(u, v). 

Remark 2.4: As q — 2, the second-order weight coincides 
with the well-known joint weight (see e.g. y_6J, 1,27] ). How- 
ever, they are different in general. For any (u, v) G F" x F" 
the joint weight of (u, v) is a 4-tuple {wo^o,wifi,wo^i,wi^i) 
with 



Wa,b 



E 



l{w{ui) = a, w{vi) = b} for a,b — 0, 1. 



From the viewpoint of group actions on sets, the joint weight 
is essentially an identification of the orbits of M„ x Af„ on 
F^ X F^ by the action ((^,C),(u,v)) ^ (^(u),C(v)). Since 
the group 7\f„ can be embedded (as a diagonal subgroup, 
which is proper for q > 3) into A/„ x Af„ by the monomor- 
phism ^ H^ {(_, ^), the partition yielded by the second-order 
weight is a refinement of the partition yielded by the joint 
weight. Therefore, when q > 3, the second-order weight 
provides more information than the joint weight. For example, 
we can determine whether two vectors are linearly independent 
by their second-order weight, but not by their joint weight. 
Suppose that the second-order weight of (u, v) G F" x F" is 
i — {'is)ses- Then u and v are linearly independent if and 
only if 

J2 1{JS>0}>1. 

seSg„ 

On the other hand, consider the following two pairs of vectors 
inF|: 

((l,2,0),(2,l,0))and((l,l,0),(2,l,0)). 

It is clear that they have the same joint weight (1, 0, 0, 2), but 
the first pair is linearly dependent and the second is linearly 
independent. 

Next, we proceed to define the second-order weight distri- 
bution and the second-order weight enumerator 

Definition 2.5: For any U,V C F" the second-order 
weight distribution of ([/, V) is defined by 

A{U,V) = |{(u,v) eUxV: W2(u,v) = i}| 

where i G Vn = {j e N^ : J^ses Js = "}. 
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Definition 2.6: For any U,V C F" the second-order 
weight enumerator of (C/, V) is a polynomial in q + 2 in- 
determinates defined by 



ueu.vev 



ieP„ 

where x = {xs)seS- 

The next four lemmas give basic properties of the second- 
order weight distribution. 

Lemma 2.7: Let U = Ui x U2 and T/ = Fi x V2, where 

Ui,ViC F;'i and L/2, ^"2 C F'^^ Then 



Wuy{^) = W^c/i,yi(x)VF[/,.v.(x). 



Proof: 



Wu,vi^) = ^ x^^("'") 



ueu.vev 



E 



^W2(ui,Vi)^W2(u2,V2) 



uie;7i,vieVi 

U2Gf2,V2GV2 

\ ^ jj.W2(ui,Vi) \ ^ J^W2(U2,V2) 

UiSC/ljViSVi U2GL'2:V2GV2 

W^(7i,Vi(x)VK"c/2,y2W. 



whenever W2(u', v') = W2(u, v). Then we have 

^w2(u,v)(F^,r;)nu e s„(w),v e s„(v)} 

^ P{u'es„(i^),v'es„(v)} 

u' ,v':W2 (u'.v')— W2 (u.v) 

= E Y. i{u'eS„(^Y),v'eS„(v)} 

u',v':W2(u',v')— W2 (u,v) 

This combined with (|7]i gives dHJ. D 

Remark 2.10: Lemma IZ91 can be further generalized to the 
case of a mapping randomized by TZ defined by ^. To this 
end, we need the concept of second-order input-output weight 
distribution, an analogue of input-output weight distribution. 
This generalization can facilitate the computation of the 
second-order weight distribution of serially concatenated codes 
with all component codes randomized by TZ. Since Lemma IZ91 
is enough for this paper, we leave this generalization to the 
reader. 

Lemma 2.11: For U,V <^ F^\ the product Wu{x)Wv{y) 
can be obtained from the second-order weight enumerator 
Wjjyi'y^) by the substitution 



xg i-j> x"^'"'^^'^^^^^^ li"^'^'^^'^^^^^^ 



ys eS 



(9) 



Lemma 2.8: 



A(F"F")=( (9-1) 



W%"F"(x) = 



^So. 



(-^-DE 



xs 



ses- 



where 



1^ rises «s!' 



D 



(4) 



(5) 



(6) 



where tt^ (i = 1, 2) is the canonical projection F^ -^ ¥q given 
by (vi,V2) '-^ Vi. As a consequence, we have 



MU)AkiV)= £ 



E 



Aiu,v). (10) 



/=0 



E 



sesii 



is=l 



*Sio=i-''*Soi='=-' 



Proof: It is clear that 

W^F,,F,(x) =a;5oo + (9- 1) E ^■5- 

This together with Lemma IZTl vields ^ and (|5]l. 



The proof is left to the reader. Note that Aj{U)AkiV) = 
coe{iWu{x)Wv{y),x^y''). 

One of the most famous results in coding theory is the 
Mac Williams identities [281. Now, we shall derive an ana- 
logue of Mac Williams identities for the second-order weight 
distribution. 

Theorem 2.12: For any V C F" we define the orthogonal 
set V^ by 



D 



Lemma 2.9: Let c e (F*)" and a e S„. Then for any V^ = <V e¥'^ : v ■ V = Y ^rV- = for all v £ V" 



u,vc f; 

Ai(c/,y) = Ai(Cc,.(C/),?c,.(F)) Vien.. (7) 

where ^c.cr is a monomial map defined by ^. Moreover, for 
any random U,V C F" 

P{u e ^„(^),v £ ^„(V)} ^ (8) 

^W2(u,v)lJ^g7J'^g j 

for all u, V G F" where S„ is a uniform random monomial 
map. 

Proof: Identity O clearly holds. As for (jSj, we note that 

P{u' e s„(w),v' e s„(v)} = P{u e s„(iY),v e s„(V)} 



Then for any subspaces U,V C. F" 

1 



W, 



u-^,v^ 



(x) 



|c/||y 



■Wuyi^K) 



(11) 



(12) 



where K is a (q + 2) x (q + 2) matrix (K5 t)s Tes defined 
by 



Ks' 



|5'|, TCS^ 
-1, T^S'-L. 



(13a) 
(13b) 



Proof: Since U and 1/ are subspaces of F" it follows 
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from Lemma lA. 1 1 that 



W, 



u^,v- 



(x) 



u'.v'SF" 



l^.(u')ly.(v')x- 



'W'"2(u'.v') 



1 



\u\\v\ ^ ^ 



X 



Applying Lemma IA.21 then gives 

1 



(u-u'+vv')x^^("''^'). 



W, 



u^y^ 



(x) 



\U\\V\ ^ 



(xK)"^("'^) 



as desired. 



D 



Remark 2.13: The set F^, as a direct product of F^, is a 
Frobenius ring because F^ is a Frobenius ring and the class of 
Frobenius rings is closed under finite direct products of ringsO 
Consequently, Theorem l2. 12l can be regarded as a consequence 
of the generalized Mac Williams identities for linear codes over 
finite Frobenius rings ll20l . ll37l . 



III. Second-Order Weight Distributions of 
Regular LDPC Code Ensembles 

Equipped with the tool established in Section|lll we proceed 
to compute the second-order weight distributions of regular 
LDPC code ensembles. 

At first, we compute the second-order weight distributions 
of two simple codes, the single symbol repetition code and 
the single symbol check code. 

Definition 3.1: A single symbol repetition map fl^^ : ¥q — >■ 
F^ with the parameter c is given by w i— >■ (t), t), • • • , v). The 
image of /^°p is called a single symbol repetition code, which 
we denote by C™p. 

Lemma 3.2: For the single symbol repetition code C^°p, 



W, 



cl°^.,c" 



= (x)=x^s„„ + (?-!) E 



sess 



F„ 



The proof is left to the reader. 

Definition 3.3: A single symbol check map f^^ 
with the parameter d is given by v M- X]i=i ^i- The kernel of 
ff^^ is called a single symbol check code, which we denote 
by Cf^. 

Lemma 3.4: For the single symbol check code C[ 



F^ 



chk 



w, 



C5''\C5 



c(x) = — 
9 



>sK 



S,S(-)\ 






where K^t is defined by (fTsT l. 

Proof: Use Theorems 12. 121 and Lemma l372l with Cf^"^ = 



^A ring R is said to be a Frobenius ring if there exists a group liomomor- 
phism / : (_R, -|-) — > C* (character of {R, +)), whose kernel contains no 
nonzero left or right ideal of R. Such a homomorphism is called a generating 
character of R.. The reader is referred to i23i Sec. 16] for background 
information on Frobenius rings. 



Example 3.5: When q — 'i. Lemma [341 gives 



Wr. 






.(x) 



^(0,0) 



2x- 



(0,1) 



2x- 



(1,0) 



2a;- 



(1,1) 



2a;- 



(0,0) 



2a;- 



(0.1) 



(1,0) 



^(1,1) 



(1,2) 



^(^^(0,0) ^(0,1) + ^^(1,0) ^(1,1) ^(1,2) 



(1,2) 
d 



'^(0,0) ^(0,1) ^(1,0) ^(1,1) 



2a;- 



(1,2) 



We are now ready to compute the second-order weight 
distributions of regular LDPC code ensembles. There are a 
few kinds of regular LDPC code ensembles |24|. We shall 
consider here two typical regular LDPC code ensembles. 
For convenience, we denote by f^^ (resp. fl°^) the n-fold 
Cartesian product of ff^"^ (resp. Z^'''')- 

The first ensemble of regular LDPC codes is due to Gallager 
|fT9l . Though it is only known as a binary regular LDPC code 
ensemble, its extension to a finite field is immediate. 

Definition 3.6: Let c, d, and n be positive integers such 



F, 



n/d 



be a random linear 



lZn/di^n{v)). 



that d divides n. Let F^^l : ¥'^ 
transformation defined by 

The regular LDPC code ensemble I, which we denote by 
^c*^! n' is defined as the intersection of c independent copies 
of the kernel of F]^^. 

According to this definition, Cj.'^J „ is the solution space of 
the random equations 

il«(v)=0, i = l,2,...,c 

where iJ'^'' is the ith independent copy of F^J^\^. In other 
words, the parity-check matrix of C^^]^ „ consists of c subma- 
trices, each being an independent copy of the transformation 
matrix of F'^^K^ (with input vectors in column-vector form). 
Since the transformation matrix of F'^^K^ contains exactly one 
nonzero entry in each column and d nonzero entries in each 
row, the resulting parity-check matrix is a {nc/d)-hy-n random 
sparse matrix with c nonzero entries in each column and d 
nonzero entries in each row, which motivates the term "regular 
low-density parity-check code ensemble". 

Theorem 3. 7: For the regular LDPC code ensemble C^^]^ „, 



w\A-(c^'^^ r'^i w 


coef([VFc5h.,cch.(x)]"/^x') 


c 


^[^A'^c,d,n->'^c,d,n)\ 


[(?)('? -1)"-^^-]'"' 


where i e ■p„. 


Proof: For any u, v G 


F^ with W2(u, v) = i. 





P{u,v G ]^erF]^^} ^^ P{u,v G S„((Cr )"/'^)} 

(b) Ai((Cg'''^)"/'^, (Cghk)n/rf) 

" ^i(F«,F^) 

(^) coef([PFccH.,ccH.(x)]"/^x') 
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where (a) follows from Definition 13.61 (b) from Lemma 12.91 
(c) follows from Lemmas 12.71 and 12.81 This together with the 
identity 

u,v:W2(u,v)— i 
u,v:W2(u,v)— i 

establishes the theorem. D 

The second ensemble of regular LDPC codes is the regular 

bipartite graph ensemble suggested by JU, ||26l . ||34l . 

Definition 3.8: Let c, d, and n be positive integers such 

that d divides en. Let F^"^^^ : F" — > Fg"' be a random linear 

transformation defined by 

The regular LDPC code ensemble II, which we denote by 
C^^l^, is defined as the kernel of i^^^i^„0 

Theorem 3.9: For the regular LDPC code ensemble C''^J„, 



where 






C^)coef([^gc..^cc..(x)]^"/^x") 



where i G Vn and ci ~ {cis)ses- 

Proof: For any u, v e F" with W2(u, v) — i, 

P{u,^eCl%} 

(b) AciiJCf^Y"/'^, (Cghk)cn/d) 

" v4,i(F™,F™) 

(^) Coef([W-cch.^cr(x)]^"/^x") 



where (a) follows from Definition 13.81 (b) from Lemma 12.91 
(c) follows from Lemmas 12.71 and 12.81 This together with the 
identity 

u,v;W2(u,v)— i 

establishes the theorem. D 

Remark 3.10: When g = 2, Lemma [J!4l and Theorems 13.71 
and 13.91 give 



F['^'A^c,d,m^cA,n)\ ~ 



[coef((ff,(x))"/^x')]' 



(i) 



c-l 



and 



P^[-^'A^c,d,m^c,d,n)\ — 



(^)coef((grf(x))^"/^x") 
(a) 



If regarding fc^n as n variable nodes (each with c sockets) and /^ J^ ;^ 
as cn/d check nodes (each with d sockets), we immediately obtain the well- 
known bipartite graph model, where the connections between variable nodes 
and check nodes are given by the uniform random monomial map Hen- 



ffd(x) = - 



•^(0,0) + ^(0,1) + ^(1,0) + ^(1,1) 



+ '^(0,0) ^(0,1) +^(1,0) ^(1,1) 



( 



^(0,0) + ^(0,1) ^(1,0) ^(1,1) 



0) " ^(0,1) ^(1,0) +^(1,1) 



Furthermore, Lemma |2. 11 1 shows that 

E[MCl%)A,{Cl%)] 

min{j,fc} 

= E 



i=0 



n \ 

^Ij — Ik — In — j — k + lJ 

X fcoef((gd(x))"/^x;^-'=+'a;^l±^^a;^) 
V ^^-^^ ^^ (0,0) (0,1) (1,0) (1,1)^ 

and 

E[A,iCi%MkiCi%)] 

Ynin{j,k} / / n \ 

El \l j-l k-l n-j-k+l) 

I 7 '^ \" 

/— \ ^c/ c{j — l) c(k~l) c{n—j — k-\-l)/ 



(0,0) 



""(ooy ^(W ii-i)' 



The second formula with j ~ k coincides with the second- 
moment formula of binary regular LDPC codes given by HI, 
EU, L32J. 



IV. Applications in Random Coding Approach 

As discussed in Section|lIl the uniformly distributed random 
mxn matrix G plays an important role in information theory 
because of the property that uG is uniformly distributed over 
F" for every nonzero u 6 F™. But it is not the end of the story. 
Theorem IB. II shows that UG is uniformly distributed over 
pnxn £qj. gygj-y invertible matrix U G F™^™. In particular, if 
m > 2, the product UG is uniformly distributed over F^^" 
for every matrix U G F^^™ of rank 2. In other words, for 
any two linearly independent vectors u, v G F™, the random 
vectors uA,vA are uniformly distributed and independent 
(see Corollary IB. 2b . In this section, we shall show that this 
property is reflected in the second-order weight distribution, 
which thus plays an important role in some well-known 
problems in coding theory and combinatorics. 

At first, for positive integers m,n,k with k < minJTO, n}, 
a random mxn matrix A is said to be k-good if UA is 
uniformly distributed over F^^" for every matrix U G F^^™ 
of rank /cO In this paper, we are only concerned with the cases 
k = 1 , 2. It is clear that the uniformly distributed random mxn 
matrix G with ni,n > 2 is 2-good and 2-goodness implies 1- 
goodness. However, a 1-good random matrix is not necessarily 
2-good and there are also other 2-good random matrices than 
G. The next example illustrates these two facts. 

^In the sequel, when speaking of a fe-good random mxn matrix, we shall 
tacitly assume that k < min{m, n}. 
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Example 4.1: Consider the matrix space F^ , which is 
identified with F| by viewing the columns of A G ¥2^^ as 
coordinate vectors relative to 1, a, cP' , where a'^ + a + 1 = 0. 
The random matrix uniformly distributed over the set 

Ax = {x{\,a,o?) : a; eFg} 

is 1-good but not 2-good. The random matrix uniformly 
distributed over the set 

A2 = {a:(l, a, c?) + 2/(1, c? .a^) : x, y £ Fg} 

is 2-good and it is clear that A2 is a proper subset of Fj^'^. 
In fact, both A\ and Ai are maximum-rank-distance (MRD) 
codes 1 13 1, nil, 105]. The reader is referred to |39| for a 
detailed investigation of the relation between fc-good random 
matrices and MRD codes. 

Next, let us compute the second-order weight distribution of 
linear code ensembles generated by a 2-good random generator 
or parity-check matrix. 

Theorem 4.2: Let A be a 2-good random to x n matrices 
with m < n. Let C^'^n and C]}^"^ be two random linear code 
ensembles generated by the generator matrix A and the parity- 
check matrix A, respectively. Then we have 



E 



Wc 



C^"„ ,c. 



- w 



rlc 12 



\nr\c 



1 



qm _i 



Efi 



Sesi 
(9" - l)(g'"-g) / 1 



-a^Sc, 



q-l 



-xs 



2'^Soo 






and 

E\W(^r\c2 (;:i-!c2 



(x)] 



(g™-l)(q'"-q) 



-Soo 



rn _ 2 

— ^t:^^- y^ (^Soo + (9 - l)a;s)" 



^2m / J 



q 



1 

2m 



XSoo + (9 - 1) Yl ^■S 



E 



W2 (uA,u'A) 



u,u' is linearly independent 






E 

u,u' is linearly independent 



W2 (uA.u'A) 



X" 



y \ v'GFJ 

+ (g™ - 1) XI g""x"^(^'°) 

veFj 

+ (g™ - 1) X X ^-"X*2(v,av) 

+ (r-l)(9"-g) E ^-'"x^^^-^^'M 

v.v'GFJ / 

= irko + (9'"-i)f E'?"'-(a^l 
^ L Vu'eF, / 

\ kGF, / 

+ ir - 1) E f E '?''^(^) 

+ (g™-l)(9"'-g)f E ?"'^(^) 
Vf.-u'eF, / 



1 



'^Soo + (9" - 1) E f ^^^00 + ^-S 



•ses,- 



+ (g™-l)(g™-g)(lx5,, + ^ 



E 



xs 



E 



Proof: Since C;/j<;„ = {uA : u G F™}, we have 
W^cri- fill- (x) 



where (a) follows from the 2-goodness of A. 

Next, since C^lf^^ = {v G F^ : Av^ = 0}, it is clear that 
C^l = {C^^)^, and then Theorem |2j2] shows that 



\c. 



Ic 12 

m.n I 



^[Wc^o2,cri<=?.(x)] =£; 



£; 



v,v'ec^'^„ X 



W2(v,v') 



|C 



rlc 12 

m.n I 



M^^. 



C'J.'^^^C?: 



,(xK) 



rlc 12 



|c,^%l 



-s 



-E 



E -' 

u.u'GFJ* 

„W2(0,0) 



W2 (uA,u' A) 



y^ ^W2(uA,u'A) 



(g"-l)(g'"-g) n 

„2m ^Soo 

+ ^^ E (a^soo + (g - i)2^sr 

SGSgo 
+ -^ ^Soo + (g - 1) E ^5 ■ 



u=0,u'5^0 
y^ -jj.W2(uA,u'A) I y^ y^ ^W2(uA,auA) 



D 

Remark 4.3: Note that the size of Ct^„ is random and it 



U5^0,u'=0 



aeF* u#0 



may be less than g™. For this reason, we give in Theorem 14.2 
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the expectation of the ratio W^ric cr'" (^)/l^m nl instead of 



Wr 



E 



^fil 



(x). Theorem 14.21 and Lemma |2. 1 1 1 show that 



Wcr^. (a;)Wc,ic (y) 



1 

r,2m 



\ni\c 12 

q 
^ -1 



2m 



1 

q 

q-l 

q 



q-l 



(g™-l)(g-l) 



jq"' ~ l){q"' - q) 



1 



-xy 

1 

— X 



q 



-V 



This is equivalent to the formula [7 Eq. (2)], in which a 
linear code is allowed to contain duplicated codewords, so 
that E{{A^{C^^^)f\ > 1. 

Now let us show that the 2-good property is reflected in the 
second-order weight distribution given by Theorem 14.21 

Proposition 4.4: Let 23 be a random linear code having the 
same average second-order weight distribution as C^^. Then 
we have 

P{u e S„(23)} = (j^™ for any nonzero u G F^' 

and 

P{u£S„(P),vGS„(23)}-g-2'n 

for any linearly independent u, v e F"0 

Proof: From Lemma l2!9l and Theorem 14.21 it follows that 
for any nonzero u G F" 

P{u G S„(23)} = P{u G S„(23), G S„(2?)} 
A„,(u,o)(F^,F^) 



Similarly, for any linearly independent u, v G F" it follows 
from Lemma 12.91 and Theorem 14.21 that 

rlc2 ^^102 ^ 



P{ugS„(P),vgS„(23)} 



^W2(u,v) V^ q t"- q J 
„-2m 



D 

Clearly, the second identity in Proposition ^. 41 reflects the 2- 
good property. The 2-good property is very useful for solving 
problems that involve pairwise or triple-wise properties of 
sequences (see ll39l ). Now based on Proposition 14.41 we shall 
provide an example of linear intersecting codes to show the 
fundamental position and potential application of second-order 
weight distribution in some problems in coding theory and 
combinatorics. 

A pair of vectors in F" is said to be intersecting if there 
is at least one position i such that their ith components are 

^There is also a similar result for C^„ , but it will be slightly complicated 
since we only know the expectation of the ratio IV^^ric (^ric {'^)/\C^„\^ 

and the fact that the coding rate of C^^ is approximately m/n with high 
probability for sufficiently large n. 



both nonzero. A linear code is said to be intersecting if its 
any two linearly independent codewords intersect. Recall that 
linear intersecting codes has a close relation to many problems 
in combinatorics, such as separating systems [[30l . qualitative 
independence |!9l, frameproof codes [10], and so on. 

Let 23 be a random linear code having the same average 
second-order weight distribution as CJj'^^„ „ with < ?7i < n. 
From Proposition 14.41 it follows that for any linearly inde- 
pendent U,V G F^\ P{U,V G S„(2?)} = q2(m-n)^ -pjjgjj 

the probability that E!„(2?) contains a given pair of non- 
intersecting and linearly independent vectors is (j2(m-n)^ ^^^ 
hence the probability that S„ (2?) contains at least one pair of 
non-intersecting and linearly independent vectors is bounded 
above by 

(2(7 - l)"g2(m-ri) ^ -2n(l-m/ri-log^(2g-l)/2) _ 

Consequently, the probability that S„(23) is not intersecting 
converges to as n -^ oo whenever the ratio 

i? = — < 1 - 1 log„(2g - 1). (cf. m Theorem 3.2]) 
n 2 ^ 

The right hand side of the inequality is the asymptotic (random 

coding) lower bound of maximum rate of linear intersecting 

codes. In order to finally relate this bound with the random 

coding rate of S„(23), we still need to study the coding rate 

of sample codes in S„(23). From Proposition 14.41 it follows 

that 

E[\E^{V)\] = V P{u G S„(23)} = r + 1 - g™-" 



uGF" 



and 



Emuv)\-E[\Euv)\]r] 

= E[\Er,iV)\']-E[\Er.{V)\f 

= ^ (P{uGS„(2?),vGS„(23)} 
u,veFj 
-P{ugS„(2?)}P{vgS„(2?)}) 

/ m-n _ 2(m-n)-j 



E 



u,vGFJ\{0} 
u, V is linearly dependent 



'(g-i)(g"-i)(i-r-") 



<q 



m+l 



Applying Chebyshev's inequality then gives 

p{||S„(2?)|-g'"|>2n(7("+i)/2| 

<p{||S„(2?)|-i?[|S„(2?)|]|>ng(™+i)/2} 



< 



Using R = m/n and the simple inequality 

ln(l+x)| <2|x| for [xl < i 
we finally obtain 



P 



-logJS„(2?)|-P 



4 (l-n_R)/2^ 1 

< -^-^ > > 1-^ 



In g J n^ 

for sufficiently large n. Roughly speaking, the coding rate of 
S„(2?) is R with high probability, provided the length n is 
large enough. 
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V. Conclusion 

We established the method of second-order weight distribu- 
tions. An analogue of Mac Williams identities for second-order 
weight distributions was proved. We computed the second- 
order weight distributions of several important code ensembles 
and discussed the application of second-order weight distribu- 
tion in random coding approach. The obtained second-order 
weight distributions are very complex, so understanding their 
significance will be our future work. 
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From Lemma lA. 11 it follows that 



{u',v')) 



{u',v')es 
Therefore we have 



u'.v'GF" 



U 



1^1, 
-1, 



v')x" 



i=ises 









S,(Mi,-Ui) 



= n E^sKs, 



•n"T(w2(u,v)) 



TeS \S£S 



as desired. 



D 



Appendix A 
Two Lemmas for The Proof of Theorem |2.12| 

Lemma A.l: Let F be a subspace of F" Then 



iv-(v') = ^E^(^-^') 



w 



vey 



where 



X(") 



A 27riTr(D)/p 



VweFo 



Tr(u) = w + v^ + • • • + U' 



p'-i 



VweFo 



(15) 
(16) 



Proof: First note that Tr(w) is an Fp-module epimorphism 
of Wq onto Fp, and hence xi'^) is a homomorphism from the 
additive group of Fg to C*. 

For a fixed v' G F^, the mapping t : y — ;■ Fg given by 
V I— >■ V • v' is an F^-module homomorphism of V into Fg, 
and hence the image set t{V) is also a vector space over Fg, 
which must be either {0} or Fg. 

If v' e V^, then x(v • v') = x(0) = 1 for all x e F, and 
hence identity (fT4l i holds. If however v' ^ V^, then 

[^E^(^'^') = - E^(^) = o- 



1^1 



vGV 



v&„ 



The proof is complete. 

Lemma A.2: For any u, v G F" 

J2 x(u ■ u' + V v')x™^("'^^') = (xK)''^("'^) 

u',v'GFj 

where K is defined by (fTsT l. 
Proof: First, we have 



D 



u',v'eF 



J2 x(u-u' + vv')x"^("''^') 

v'GFj 

n 

u'.v'eFj i=l 
n 

n 
"IIE^S E Xiiu^,V^)■{u',v')). 

4=1 ses {u',v')es 



Appendix B 

Properties of Uniformly Distributed Random 

Matrices 

Theorem B.l: Let G be a random rn x n matrix uniformly 
fmxn Qj^ all m X n matrices over Fg. 



(14) distributed over the set 



Then for any invertible matrix U G F™^™, the product UG 
is uniformly distributed over F™^". 

Proof: For any matrix U G F™^™, we denote by U* 
the mapping F™^" -^ pnxn gjygjj by X h-> UX. It is clear 
that U* is a surjective linear transformation for any invertible 
matrix U G F™^™, so that UG is uniformly distributed over 

A corollary follows immediately. 

Corollary B.2 (cf JT) and the references therein): Let G 
be a random m x n matrix uniformly distributed over the 
set F'"^" of all m x n matrices over Fg. Then for any 

x,x' gF™ and y,y' G F'^\ 



P{xG-y,x'G = y'} 

i{y = o,y' = o}, 

<?-"l{y = 0}, 
q-^l{y' = 0}, 
9-"l{y' = ay}, 



X = x' = 
X = 0, x' 7^ 
X 7^ 0, x' = 

X 7^ 0, x' = ax with a G F* 
X, x' are linearly independent. 
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